AMF: Aggregated Mondrian Forests for Online Learning

Abstract Random forest (RF) is one of the algorithms choice in many supervised learning applications, be it classification or regression. The appeal such tree-ensemble methods comes from a combination several characteristics: remarkable accuracy variety tasks, small number parameters to tune, robustness with respect features scaling, reasonable computational cost for training and prediction, their suitability high-dimensional settings. most commonly used RF variants, however, are ‘offline’ algorithms, which require availability whole dataset at once. In this paper, we introduce AMF, an online algorithm based on Mondrian Forests. Using variant context tree weighting algorithm, show that possible efficiently perform exact aggregation over all prunings trees; particular, enables obtain truly parameter-free competitive optimal pruning tree, thus adaptive unknown regularity regression function. Numerical experiments AMF strong baselines large datasets multi-class classification.